Unmanned mobile robot and software for clinical examination and treatment

ABSTRACT

A method for photographing at least a portion of a subject is disclosed, the method comprising: generating a photography scheme, the photography scheme comprising a set of photography control points, each of the photography control points comprising: a location of the platform relative to the subject; an orientation of the platform relative to the subject; and one or more photography parameters; determining a location and an orientation of the platform carrying the imaging system; navigating the platform to each of the photography control points and operating the imaging system to capture an image of the subject at each of the photography control points according to the associated photography parameters. A method of administering photodynamic therapy to a subject is also disclosed.

REFERENCE TO RELATED APPLICATIONS

This application claims priority from U.S. Patent Application Ser. No.62/814,175, entitled DRONE AND SOFTWARE FOR IMAGING SKIN, filed Mar. 5,2019 which is hereby incorporated herein by this reference in itsentirety for all purposes. For purposes of the United States of Americathis application claims the benefit of U.S. Patent Application Ser. No.62/814,175, entitled DRONE AND SOFTWARE FOR SKIN IMAGING, filed Mar. 5,2019.

TECHNICAL FIELD

This application relates to systems and methods for imaging and/ortreating the surface of a subject, for example a subject's skin. Exampleembodiments provide systems and methods for imaging and treating using aself-propelled apparatus, for example an unmanned aerial vehicle (UAV).

BACKGROUND

There are many applications that require imaging a subject. For example,many medical examinations require imaging a patient's body or a portionthereof. Medical applications of body imaging include diagnosis andmonitoring of conditions afflicting a patient's skin, eyes, mouth, ornails. Furthermore, many of these afflictions require monitoring over aperiod of time to diagnose and treat.

One condition benefiting from monitoring over time is skin cancer.Furthermore, early diagnosis of skin cancer may improve patient outcome.Skin screening is one method to achieve early diagnosis of skin cancer.Performed regularly, self-examination can alert an individual to changesin the skin and aid in the early detection of skin conditions anddiseases. However, naked eye examination lacks the sensitivity requiredfor early-stage detection of some skin conditions and diseases, forexample skin cancer. Furthermore, differences in imaging conditions, forexample differences in lighting between different imaging sessions, maylimit the utility of such monitoring.

To diagnose and treat conditions and diseases, dermatologists and otherhealth professionals may systemically check the entire surface of theskin, hair, and nails, and especially areas exposed to the sun. Skinlesions (e.g. parts of the skin that have abnormal appearance comparedto the skin around them) and hair and nail features may be recorded byhand by plotting a full-body chart or by taking a series of images.

Total body photography (TBP) is the process of imaging skin, hair, andnails to detect, monitor, diagnose, and treat conditions and diseases.TBP may be used to measure other metrics, including, but not limited tobody shape for cosmetic and/or fitness and/or health applications.

Manually capturing images is both resource intensive and is susceptibleto errors. Images must be properly documented and analyzed to optimizediagnosis and treatment. Incongruities in, for example, lighting, theangle that the image is acquired at, etc. may impact the quality ofimages and affect detection and diagnosis.

TBP systems are known. Some systems employ numerous cameras positionedto surround a patient and simultaneously capture images. Other systemsemploy multiple cameras positioned to simultaneously capture images of asection of a patient's body. Such conventional systems are typicallybulky, expensive, and require a dedicated space and personnel tooperate. Further, depending on the position and angle of the camerasrelative to the patient's body, the quality of the acquired images maybe affected, thereby complicating detection and diagnosis. Furtherstill, since a patient's body dimensions change and the patient'spositioning relative to the cameras is difficult to replicate over time,it is difficult to reproduce the multiple variables that impact theacquisition of consistent images (e.g. the position of the patient'sbody or body segment relative to the camera, the distance of thepatient's body or segment from the camera, the orientation of thecamera, lighting, etc.). Thus, it is difficult to reproduce high qualityand consistent images of skin, hair, and nail segments that are neededto monitor skin, hair, and nail features over time.

Skin conditions may be treated with photodynamic therapy, wherein alesion is illuminated with light of a certain intensity for a period oftime. Treating skin conditions with photodynamic therapy poses similardifficulties to imaging of skin. In particular, it is difficult toaccurately administer photodynamic therapy, and consistently administerphotodynamic therapy during multiple treatment sessions.

There is a general desire for an imaging and/or treatment system capableof producing high quality, reproducible images, and/or administeringphotodynamic therapy.

The foregoing examples of the related art and limitations relatedthereto are intended to be illustrative and not exclusive. Otherlimitations of the related art will become apparent to those of skill inthe art upon a reading of the specification and a study of the drawings.

SUMMARY

The following embodiments and aspects thereof are described andillustrated in conjunction with systems, tools and methods which aremeant to be exemplary and illustrative, not limiting in scope. Invarious embodiments, one or more of the above-described problems havebeen reduced or eliminated, while other embodiments are directed toother improvements.

One aspect of the invention provides a method of photographing at leasta portion of a subject with a platform carrying an imaging system, themethod comprising: generating a photography scheme, the photographyscheme comprising a set of photography control points, each of thephotography control points comprising: a location of the platformrelative to the subject; an orientation of the platform relative to thesubject; and one or more photography parameters; determining a locationand an orientation of the platform carrying the imaging system;navigating the platform to each of the photography control points andoperating the imaging system to capture an image of the subject at eachof the photography control points according to the associatedphotography parameters.

One aspect of the invention provides an imaging system comprising: anunmanned aerial drone, the drone comprising: a drone body; four rotorsmounted to the drone body; a digital camera mounted to the drone body; alight source mounted to the drone body; a laser sensor mounted to thedrone body; a drone transceiver mounted to the drone body; a dronecomputer mounted to the drone body, the drone computer configured to:control the four rotors to navigate the drone; control the digitalcamera to capture one or more digital images; control the light sourceto emit light; receive data from the laser sensor; and transmit andreceive data via the drone transceiver; three GPS receivers, whereineach of the GPS receivers is configured to receive a signal from thedrone transceiver; a controller, the controller comprising: a memorystoring at least one previous image of a subject; a controllertransceiver configured to communicate with the drone transceiver and thethree GPS receivers; wherein the controller is configured to control thedrone to: control the rotors to navigate the drone about a subject;control the rotors to orientate the digital camera towards the subject;control the light source to illuminate the subject; control the digitalcamera to take one or more images of the subject; and store the one ormore images of the subject in the memory.

One aspect of the invention provides a method of administeringphotodynamic therapy to a subject with a platform carrying aphotodynamic treatment system, the method comprising: receiving aphotodynamic therapy prescription, the photodynamic therapy prescriptioncomprising: a therapy region corresponding to an area of the subject;and a therapy light dose; generating a photodynamic therapy scheme atleast in part based on the photodynamic therapy prescription; whereinthe photodynamic therapy scheme comprises a set of photodynamic controlpoints, each of the photodynamic control points comprising: a locationof the platform relative to the subject; an orientation of the platformrelative to the subject; an illumination intensity; and an illuminationtime; and navigating the platform to each of the photodynamic controlpoints and controlling the photodynamic treatment system to illuminatethe subject with light of the illumination intensity and for theillumination time of each of the photodynamic control points.

In addition to the exemplary aspects and embodiments described above,further aspects and embodiments will become apparent by reference to thedrawings and by study of the following detailed descriptions.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiments are illustrated in referenced figures of thedrawings. It is intended that the embodiments and figures disclosedherein are to be considered illustrative rather than restrictive.

FIG. 1A depicts a system for imaging a subject according to an exampleembodiment.

FIG. 1B depicts a system for administering photodynamic therapy to asubject according to an example embodiment.

FIG. 2A depicts a system according to an example embodiment.

FIG. 2B depicts a method for imaging a subject according to an exampleembodiment.

FIGS. 3A to 3E depict an unmanned aerial drone (UAV) according to anexample embodiment.

FIG. 3F depicts an unmanned aerial drone (UAV) according to an exampleembodiment.

FIGS. 4A to 4D depict a UAV according to another example embodiment.

FIGS. 5A to 5C depict a localization system according to an exampleembodiment.

FIGS. 6A and 6B depict an indoor-GPS according to an example embodiment.

FIG. 6C depicts a 3D reconstruction method according to an exampleembodiment.

FIGS. 7A and 7B depict an unmanned ground vehicle (UGV) according to anexample embodiment.

FIG. 8 depicts a circular stand according to an example embodiment.

FIGS. 9A and 9B depict a flash light case according to an exampleembodiment.

DESCRIPTION

Throughout the following description specific details are set forth inorder to provide a more thorough understanding to persons skilled in theart. However, well known elements may not have been shown or describedin detail to avoid unnecessarily obscuring the disclosure. Accordingly,the description and drawings are to be regarded in an illustrative,rather than a restrictive, sense.

Unless the context dictates otherwise, the term “optical element” (asused herein) refers to a lens, a filter, an optical film, a diffuser, ora polarizer.

Unless the context dictates otherwise, the term “diffuser” (as usedherein) refers to a filter that diffuses or scatters light in somemanner. A diffuser may be applied to provide soft light and/or toachieve a more uniform light distribution.

Unless the context dictates otherwise, the term “polarizer” (as usedherein) refers to an optical filter that can convert a beam of light ofundefined or mixed polarization into a beam of well-definedpolarization.

Unless the context dictates otherwise, the term “linear polarizer” (asused herein) refers to a polarizer that selectively passes or creates alinearly-polarized electromagnetic wave (e.g. a linearly-polarized lightwave). The direction of the electric field of the electromagnetic waveis aligned parallel to a polarization direction or ‘polarization axis’of the polarizer.

Unless the context dictates otherwise, the term “circular polarizer” (asused herein) refers to a polarizer filter that selectively passes and/orcreates a circularly-polarized electromagnetic wave. In acircularly-polarized wave a direction of the electric component of theelectromagnetic wave changes in a rotary manner along the direction ofpropagation. Circular polarization can be either clockwise orcounterclockwise.

Unless the context dictates otherwise, the term “cross polarization”refers to the polarization of light in an orthogonal direction to thepolarization of light being discussed.

Unless the context dictates otherwise, “focal length” (as used herein)refers to the distance between a lens and a focal point of an opticalsystem, wherein the lens converges parallel rays of light into theoptical system's focal point. The focal length of an optical system is ameasure of how strongly the system converges or diverges light. A systemwith a shorter focal length has greater optical power than one with alonger focal length since the system with the shorter focal length isable to bring light rays into focus in a shorter distance.

FIG. 1A depicts a system 100 for imaging a subject, for example by totalbody photography (TBP) of the subject. System 100 comprises: imagingsystem 110, platform 120, guidance system 130, and analysis system 140.Imaging system 110 is carried by platform 120. Guidance system 130controls platform 120. Imaging system 110 captures images and providesthe images to analysis system 140. Analysis system 140 processes theimages provided by imaging system 110.

Imaging system 110 comprises one or more digital cameras. Exampleembodiments of digital cameras include:

-   -   a digital single-lens reflex (DSLR) camera;    -   a digital camera of a tablet computer, for example an Apple™        iPad;    -   a digital camera of a smartphone, for example an Apple™ iPhone;        and    -   any other portable digital camera.

Where imaging system 110 comprises a smartphone or a tablet computer,platform 120 may comprise a mount to retain the smartphone or tabletcomputer. In some embodiments, platform 120 may comprise a lightdeflector, for example a mirror or a prism, to direct light to a digitalcamera of the smartphone or tablet computer. The light deflector may beconfigured to direct light perpendicular to the digital camera of thesmartphone or tablet computer.

In some embodiments, the smartphone or tablet computer may be mounted toplatform 120 with a face of the smartphone or tablet computer facingupward from the ground or downward towards the ground. Mounting asmartphone or tablet computer to platform 120 with a face of thesmartphone or tablet computer facing upward from the ground or downwardtowards the ground may improve the stability of platform 120. Platform120 may comprise a light deflector to direct light into the camera ofthe smartphone or tablet computer. The light deflector may direct lighttraveling horizontal to the ground into the camera of the smartphone ortablet computer.

In some embodiments, platform 120 comprises a propulsion system and ispartially or entirely self-propelled by the propulsion system. Whereplatform 120 comprises a propulsion system, guidance system may 130control the propulsion system of platform 120. In such embodiments,platform 120 is at least partially controlled by guidance system 130controlling the propulsion system of platform 120.

Example embodiments of platform 120 which comprise a propulsion systemand are at least partially self-propelled include:

-   -   an unmanned aerial vehicle (UAV);    -   an unmanned ground vehicle (UGV);    -   a motorized circular stand; and    -   any motorized device capable of carrying an imaging system.

In some embodiments, platform 120 is partially or entirely propelled bya user. Where platform 120 is at least partially propelled by a user,guidance system 130 provides human perceptible instructions to the userfor controlling platform 120. Human perceptible instructions provided byguidance system 130 may include audio and/or visual instructions, forexample audio cues, light cues, synthesized speech, pre-recordedmessages, text instructions, vibration feedback, and the like.

Examples of platform 120 which are at least partially propelled by auser include:

-   -   a flash light case;    -   a portable stand; and    -   any device which may be manually manipulated by a user.

Imaging system 110 may further comprise a light source, for example oneor more light emitting diodes, incandescent lamps, and/or fluorescentlamps. The light source of imaging system 110 may further comprise oneor more filters configured to selectively transmit light of a certainpolarity and/or spectrum. The filters may comprise one or morepolarizing filters, and/or one or more optical filters. The light sourceand any other optical elements of imaging system 110 may be used byimaging system 110 to capture images, for example to illuminate asubject to photograph.

In some embodiments, platform 120 is integrated with imaging system 110,for example a UAV or a UGV with an integrated camera. In someembodiments, imaging system 110 is removably mounted to platform 120,for example a a UAV or a UGV with a removably mounted smartphone.

Where a propulsion system of platform 120 is at least partiallycontrolled by guidance system 130, guidance system 130 may comprise oneor more modules which control the movement, position, and/or orientationof platform 120. In some embodiments, guidance system 130 is integratedwith imaging system 110. For example, imaging system 110 may comprise asmartphone and one or more modules of guidance system 130 may beimplemented by the smartphone.

Where one or more modules of guidance system 130 are implemented by asmartphone, the smartphone may be communicatively coupled to theplatform by a wired interface, for example a Lightning™ or USB cable.The modules of guidance system 130 may be downloaded to the smartphoneby downloading a mobile app. For example, a user may access an app storefrom the smartphone, and then download and run a mobile app. A mobileapp is a software application designed to run on a smartphone. A mobileapp may be downloaded from an app store, which is an online database ofmobile apps. The smartphone may comprise a memory storing the mobileapp.

Guidance system 130 may control platform 120 to:

-   -   translate platform 120 by a certain distance in a certain        direction;    -   roll platform 120 by a certain angle in a certain direction;    -   pitch platform 120 by a certain angle in a certain direction;        and/or    -   yaw platform 120 by a certain angle in a certain direction.        By translating, rolling, pitching, and yawing platform 120,        guidance system 130 may navigate platform 120 to any position        and/or orientation.

Where platform 120 is at least partially controlled by a user, guidancesystem 130 may comprise one or more systems which provide instructionsto a user controlling platform 120. Instructions provided by guidancesystem 130 to control platform 120 may include:

-   -   an instruction to translate platform 120 by a certain distance        in a certain direction;    -   an instruction to rotate platform 120 by a certain angle in a        certain direction;    -   an instruction to pitch platform 120 by a certain angle in a        certain direction; and/or    -   an instruction to yaw platform 120 by a certain angle in a        certain direction.

Where instructions provided by guidance system 130 to a user of platform120 include audio cues, providing the instructions may comprise playinga pre-recorded message, for example a message such as “lower theplatform by one meter”, or “rotate the platform around the subject by 45degrees”.

Where instructions provided by guidance system 130 include visual cues,platform 120 may comprise one or more visual outputs, for example aliquid crystal display (LCD) or a set of LEDs arranged around aperiphery of platform 120. Where platform 120 comprises a visualdisplay, guidance system 130 may display instructions via the visualdisplay. Example instructions displayed by the visual display mayinclude:

-   -   text instructing a user to move, rotate, pitch and/or yaw        platform 120 about a subject;    -   diagrams depicting moving, rotating, pitching, and/or yawing        platform 120 about a subject; and/or    -   videos depicting moving, rotating, pitching, and/or yawing        platform 120 about a subject.

Where platform 120 comprises a smartphone or tablet computer, audioinstructions may be provided by a speaker of the smartphone and/ortablet computer, and visual instructions may be provided by a display ofthe smartphone and/or tablet computer.

Guidance system 130 may also be in communication with imaging system 110and control imaging system 110 to:

-   -   capture a digital image with a digital camera of imaging system        110;    -   set one or more photography parameters of imaging system 110,        for example one or more of an aperture size, shutter speed, ISO        sensitivity, and focal length of a digital camera of imaging        system 110; and/or    -   set an intensity and/or spectrum of light emitted by a light        source of imaging system 110.

To control platform 120 and/or to generate instructions for a user tocontrol platform 120, guidance system 130 may determine and/or store:

-   -   a layout of markers (described below);    -   a current position and/or orientation of platform 120;    -   a current position and/or orientation of a subject;    -   a current position and/or orientation of a subject relative to a        current position and/or orientation of platform 120;    -   a current position and/or orientation of one or more objects        relative to a current position and/or orientation of platform        120; and/or    -   a previous position and/or orientation of platform 120 relative        to a subject.

Guidance system 130 may comprise one or more modules including:

-   -   a localization module configured to determine a location and/or        an orientation of platform 120;    -   a navigation module configured to determine a photography scheme        (described below) for platform 120 and imaging system 110;    -   an obstacle avoidance module configured to control platform 120        to avoid one or more obstacles;    -   a skeletal detection module configured to generate a skeletal        map of a subject;    -   a face detection module configured to determine a position        and/or orientation of a face of a subject;    -   a face recognition module configured to identify a face of a        subject; and/or    -   an image stabilization module configured to stabilize a digital        camera of imaging system 110.

Guidance system 130 may comprise one or more inputs, for example one ormore sensors. Examples of sensors comprising guidance system 130include:

-   -   Light Detection and Ranging (LIDAR) sensors;    -   infrared range finder sensors;    -   digital cameras;    -   inertial measurement (IMU) sensors such as gyroscopes and        accelerometers;    -   ultrasonic range finder sensors;    -   RADAR;    -   GPS;    -   electromagnetic sensors;    -   barometric pressure sensors; and/or    -   optical flow sensors.        In some embodiments, one or more digital cameras of imaging        system 110 may also be used by guidance system 130 to generate        inputs for guidance system 130.

Images captured by imaging system 110 are transmitted to analysis system140. Analysis system 140 comprises one or more modules which may receiveand/or analyze images from imaging system 110. Examples of modules ofanalysis system 140 include:

-   -   a 3D model construction module;    -   a lesion/spot analysis module; and/or    -   an automated lesion/spot detection module.

FIG. 1B depicts a system 102 for administering photodynamic therapy to asubject. System 102 comprises: imaging system 110, platform 120, andguidance system 130, similar to system 100 described above.

System 102 further comprises treatment system 150. Treatment system 150comprises one or more light sources configured to emit light foradministering photodynamic therapy. In some embodiments, treatmentsystem 150 comprises one or more light emitting diodes (LEDs) foradministering photodynamic therapy.

In some embodiments, treatment system may be configured to administerphotodynamic therapy according to a photodynamic therapy scheme.Treatment system 150 may be configured to generate the photodynamictherapy scheme at least in part based on:

-   -   a location of platform 120;    -   an orientation of the platform 120;    -   a location of the subject;    -   an orientation of the subject; and    -   a photodynamic therapy prescription.

The photodynamic therapy prescription may comprise a photodynamictherapy region and a photodynamic therapy light dose. The photodynamictherapy region may correspond to a region of a subject, for example alesion of a subject.

The photodynamic therapy scheme generated by treatment system 150 maycomprise a set of photodynamic control points, wherein each of thephotodynamic control points comprises:

-   -   a location of platform 120;    -   an orientation of platform 120;    -   an illumination intensity; and    -   an illumination time.

The photodynamic control points may be generated by treatment system 150by first determining the location and the orientation of platform 120required to illuminate the photodynamic therapy region. Once thelocation and the orientation of platform 120 required to illuminate thephotodynamic therapy region is determined, treatment system 150 maydetermine the illumination intensity and illumination time required todeliver the photodynamic therapy prescription to the photodynamictherapy region from the location and the orientation of platform 120.

Where platform 120 is at least partially self-propelled, treatmentsystem 150 may control platform 120 and direct platform 120 to each ofthe photodynamic therapy control points and control the light source oftreatment system 150 to illuminate the photodynamic therapy region ofthe subject with light of the illumination intensity and for theillumination time.

An aspect of the invention provides an UAV capable of acquiring highresolution two-dimensional (2D) and/or three-dimensional (3D) bodysurface images, including images of skin, hair, and nails. The UAVprovides a low-cost solution for TBP that is suitable for indoor use atskin clinics, medical offices, hospitals, pharmacies, home, etc.

An unmanned aerial vehicle (UAV) (also known as a drone) is an aircraftwithout an onboard human operator. An UAV is typically one component ofan UAV system, which includes the UAV, a controller, and a communicationsystem between the UAV and the controller. An UAV may operate underremote control by a human operator or autonomously by one or moreonboard computers.

Images may be acquired by the UAV in 2D or 3D. In some embodiments, theUAV is used to acquire 3D body surface images directly in 3D with a 3Dcamera, or by merging position-known 2D images acquired by navigatingthe UAV around an object to be imaged. The images may be a series of 2Dor 3D images, which may be combined to form a 3D representation of theimaged body.

In some embodiments, the UAV includes a digital camera for acquiringimages. The UAV may include one or more sensors, LED cross-polarizedlighting, and/or an onboard computer system (i.e. hardware and software)capable of real-time image acquisition, storage, and/or analysis. Insome embodiments, the UAV may transmit images wirelessly to an externalcomputer system to analyze the images acquired by the UAV.

In some embodiments, the position and/or location of the UAV may bedetermined in real-time. The UAV may be maneuvered automatically tospecific locations to capture desired images. In this way, the UAV iscapable of taking reproducible images. Such images are useful for TBP.When such images are taken over time, comparisons between such imagesmay be further useful for TBP.

In some embodiments, the UAV includes means for stabilizing the UAV. Forexample, the UAV may comprise one or more stability sensors such asgyroscopes and/or accelerometers to measure the tilt, rotation, and/orpitch of the UAV. Such measurements of tilt, rotation, and/or pitch maybe used to stabilize the UAV. By stabilizing the UAV, high qualityimages may be acquired. Such images may be useful for TBP.

In some embodiments, the UAV may be used to acquire full-body orpartial-body surface images. Such images may be used and analyzed forautomated screening of skin conditions, for example automated screeningfor skin cancer, pigmented lesions, and/or vascular lesions. Inaddition, the UAV may be used to analyze other skin conditions, such asacne, rashes and inflammatory diseases such as psoriasis or eczema. TheUAV may be used to estimate the size and coverage of the disease area aswell as to estimate the depth of the condition (for example wrinkles,raised lesions, wounds, etc.), to monitor a condition on different bodyparts, and/or to monitor treatment progress. The UAV may be used forcosmetic and/or plastic surgery applications and/or may be used bydermatologists, surgeons, general practitioners, nurses, photographers,and consumers/patients.

In some embodiments, the UAV may be used by a user to acquire anoverview image of a patient's body or body part. In some embodiments,the UAV may be used to identify and image a body or body part, detectskin lesions in the acquired image, and label and place the lesions on a2D or 3D body map.

In some embodiments, the UAV may be used to automatically identify areasof interest of the skin such as acne, rashes, psoriasis, eczema andwounds, and place them on a 2D or 3D body map for labelling, archivingand monitoring over time.

In some embodiments, the UAV may be used to analyze skin lesions on areal-time basis. Onboard and/or external computer systems may be used toinstruct the UAV to reimage and/or get closer to a lesion to take higherquality images.

In some embodiments, the UAV is equipped with a dermoscope to takedermoscopy images. The dermoscopy images may be any combination ofpolarized/non-polarized/cross-polarized images.

FIG. 2A depicts an embodiment of imaging system 110 and platform 120comprising unmanned aerial vehicle (UAV) 200 and controller 202.

UAV 200 comprises body 204 and rotors 206. UAV 200 is lifted andpropelled by rotors 206.

UAV 200 further comprises digital camera 208 and light source 210. Lightsource 210 is configured to illuminate a subject, and digital camera 208is configured to capture one or more digital images of the illuminatedsubject. In some embodiments, light source 210 emits polarized light.

UAV 200 further comprises one or more sensors 212, transceiver 214, andonboard computer 216. Computer 216 comprises a memory and a processor.Computer 216 is communicatively coupled to digital camera 208, lightsource 210, sensors 212, and transceiver 214.

Controller 202 comprises transceiver 218 and computer 220. Computer 220comprises a processor and a memory storing software to be executed bythe processor. UAV transceiver 212 and transceiver 218 are configured towirelessly communicate with each other, for example by the MAVLink™communication protocol.

Sensors 212 may comprise one or more devices capable of measuring aparameter of the environment proximate UAV 200, for example one or moreof: LIDAR sensors, infrared range sensors, digital cameras, ultrasonicrange sensors, accelerometers and an indoor-GPS transmitter (describedbelow).

Computer 216 is configured to receive sensor data from sensors 212.Computer 216 may control rotors 206 in part based on sensor datareceived from sensors 212. For example, computer 212 may control rotors206 to navigate UAV 200 to avoid an obstacle, for example to avoid awall, ceiling, floor, object or person.

Computer 216 is configured to control transceiver 214 to transmit sensordata to controller 202 via transceiver 218. Computer 220 may beconfigured to receive sensor data from transceiver 218.

Computer 220 is configured to control UAV 200, and control transceiver218 to transmit commands to UAV 200 via transceiver 214. Computer 216may receive commands from transceiver 214 and control rotors 206 tonavigate UAV according to the command.

Computer 220 may generate commands for controlling UAV 200 based in parton one or more of: received sensor data, stored images, and user input.For example, computer 220 may generate one or more commands to:

-   -   control rotors 206 to navigate UAV about a subject, and orient        light source 210 and digital camera 208 relative to the subject;    -   control light source 210 to illuminate the subject; and    -   control digital camera 208 to photograph the subject.

Computer 216 may be configured to control transceiver 214 to transmitdigital images generated by digital camera 208 to controller 202 viatransceiver 218. Computer 216 may be configured to store digital imagesgenerated by digital camera 208 in a memory, for example a flash memorycard.

FIG. 2A depicts an example method 201 performed by UAV 200 andcontroller 202 for photographing a subject. Prior to performing method201, the subject may be positioned in a pre-determined location. Forexample, the subject may be a person, and the person may be instructedto stand in a specified location with a specified posture. For example,the person may stand upright with their feet placed on a specificlocation and with their hands grasping handles fixed proximate thespecific location.

Method 201 comprises:

-   -   step 232: determine a location and an orientation of UAV 200;    -   step 234: determine a location and an orientation of a subject;    -   step 236: receive one or more images of the subject;    -   step 238: generate a photography scheme; and    -   step 240: control UAV 200 according to the photography scheme.

The location of UAV 200 may be determined in step 232 for example by UAVphotographing at least one marker in a network of markers, and computer220 determining the location of UAV 200 from the photographed marker, asdescribed below. The position of UAV 200 may be stored in the memory ofcomputer 220. The position of UAV 200 may be represented by a locationvector representing a direction and distance from a certain location.For example, one marker in the network of markers may be designated as aprime marker, and all other location vectors may represent a directionand distance from the prime marker.

The orientation of UAV 200 may be determined in step 232 by determiningan orientation of the photographed marker. To determine the orientationof a photographed marker, computer 220 may determine a rotation of thephotographed marker from an orientation of a known marker. For example,computer 220 may determine that a photographed marker is rotated acertain degree from a known marker. If digital camera 208 is mounted ata known position on UAV 200, then the orientation of UAV 200 can bedetermined from the certain degree of rotation of the photographedmarker.

For example, a known marker may have a known orientation relative to theprime marker. Computer 220 may determine a rotation of a photographedmarker relative to the known marker, and thereby determine a rotation ofthe photographed marker relative to the prime marker. Computer 220 maythen determine a rotation of UAV 200 relative to the prime marker fromthe rotation of the photographed marker relative to the prime marker.

The location and orientation of the subject may be determined in step234 by positioning and orientating the subject in a pre-determinedlocation, and storing the pre-determined location of the subject in thememory of computer 220. For example, where the subject is a person, theperson may be positioned by standing on a certain marker, designated asubject marker, in the network of markers. Computer 220 may store theposition of the subject marker as a location vector representing adirection and a distance from the prime marker.

The person may be positioned at the subject marker in a certainorientation by having the person assume a certain posture at the subjectmarker. For example, the person may be instructed to: stand, sit or liedown, and/or grasp a handle fixed proximate the subject marker, and/orarrange their limbs in a certain manner. For example, the person may beinstructed to stand upon the subject marker with their body erect andwith their hands at their sides.

The person to be photographed may be instructed to assume a posture atthe subject marker corresponding to a position and an orientation of arepresentative person. Computer 220 may store a position and anorientation of the representative person as a three dimensional (3D)model. The 3D model of the representative person may represent theposition of major features of the representative person. For example,the 3D model may include location vectors for each of the representativeperson's major body parts. Such major body parts may include therepresentative person's head, chest, arms and legs. The body partlocation vectors may be represented as location vectors relative to theprime marker.

The one or more images of the subject may be received in step 236 bycomputer 220 retrieving the images from the memory of computer 220. Suchimages may have been captured previously by UAV 200. The images mayinclude associated metadata, for example one or more photographyparameters (described below) and a location and/or orientation of UAV200 used to capture an image. The location and/or orientation of UAV 200may be represented as a location vector indicating a direction anddistance from the prime marker.

Step 238 may comprise generating a photography scheme based at least inpart on:

-   -   the location of UAV 200;    -   the orientation of UAV 200;    -   the location of the subject;    -   the orientation of the subject; and    -   the one or more images of the subject.

In some embodiments, the photography scheme comprises a set ofphotography control points, each of the photography control pointscomprise:

-   -   a location of UAV 200;    -   an orientation of UAV 200; and    -   one or more photography parameters.

The one or more photography parameters may include one or more of: ashutter speed; an aperture size; a focal length; an ISO sensitivity; andan illumination level.

To generate the photography scheme, computer 220 may determine a numberof images required of the subject. For example, computer 220 maydetermine that thirty images of the subject are required, where tenimages are taken at a first elevation, ten images are taken at a secondelevation greater than the first elevation, and ten images are taken ata third elevation greater than the second elevation. Each of the tenimages at each elevation may be separated by an equal angle about thesubject. For example: a first image may be taken at a starting position,a second image may be taken at a second position 36° around the subjectfrom the starting position, a third image may be taken at a thirdposition 72° around the subject from the starting position, and so onwith each image being taken at a position (n×36°) around the subjectfrom the starting position, where n is an index identifying the imagewith 0≤n≤9 in this example.

Once computer 220 has determined the number of images of the subjectrequired, computer 220 may determine the photography control pointsrequired to capture each of the required images. For example, computer220 may determine a position and an orientation of drone 200 required tocapture each of the thirty images referenced above. To determine theposition and orientation of drone 200 required to capture each image,computer 220 may determine a distance from the subject for each image.The distance from the subject for each image may be represented as animage vector with a direction and distance from the subject marker. Theset of image vectors may then be used to generate the set of photographycontrol points specifying the location and orientation of UAV 200required to capture each of the required images.

Computer 220 may generate one or more photography parameters for each ofthe photography control points. For example, computer 220 may set afocal length and an aperture size of a photography control point tocapture a certain depth of field a certain distance from drone 200. Thedistance of the depth of field from drone 200 may be approximately equalto a distance from the subject to drone 200 when drone 200 is at thelocation and orientation of the respective photography control point.The depth of field may be approximately equal to the depth of thesubject. In some embodiments the depth of field may be 1 meter.

Computer 220 may also set a shutter speed and ISO sensitivity for eachof the photography control points. Computer 220 may set the shutterspeed and ISO sensitivity of a photography control point as a functionof the focal length and aperture size of the photography control point.For example, computer 220 may set the shutter speed and ISO sensitivityof a photography control point to produce a certain exposure level. Anexposure level approximates the amount of light reaching digital camera208, which affects the brightness of an image captured by digital camera208.

Computer 220 may modify the photography control points based on one ormore previous images of the subject. For example, computer 220 maydetermine that a previous image of the subject was taken by drone 200 ata certain location and with a certain orientation. Determining thelocation and orientation of drone 200 used to capture the previous imagemay comprise reading metadata associated with the previous image.Computer 220 may then select one of the photography control points withthe nearest position and orientation to the position and orientation ofthe previous image, and modify the location and orientation of theselected photography control point to more closely match the positionand orientation of the previous image.

Computer 220 may modify the photography control points based on one ormore previous images of the subject by adding a photography controlpoint. The added photography control point may have the same location,orientation, and photography parameters as the location, orientation,and photography parameters of a previous image of the subject.

In some embodiments, a user may select a previous image of the subjectto be reproduced. The user may select the previous image from a databaseof previous images using an interface of computer 220. Computer 220 maythen add a photography control point with the same location,orientation, and photography parameters as the location, orientation,and photography parameters of the selected image of the subject.

Controlling UAV 200 according to the photography scheme in step 240 maycomprise controller 202 determining commands to navigate UAV 200 betweena current position and orientation of UAV 200 and a position and anorientation of one of the photography control points, and controllingUAV 200 to navigate UAV 200 to one of the photography control points.The commands determined by controller 202 may include commands tooperate the rotors of UAV 200 to navigate UAV 200. For example,controller 202 may increase power to one or more of the rotors of UAV200 to move UAV 200 in a certain direction.

Once controller 202 determines that UAV 200 is at one of the photographycontrol points, controller 202 may control digital camera 208 to capturea photograph according to one or more photography parameters associatedwith the one of the photography control points.

Computer 220 may modify one or more photography control points based onsensor data acquired while controlling UAV 200 according to thephotography scheme in step 240. For example, sensors 212 may comprise aLIDAR sensor configured to measure a distance from UAV 200 to thesubject. Once UAV 200 is at a photography control point, the LIDARsensor may determine a distance between UAV 200 and the subject.Computer 220 may then modify one or more photography parameters of thephotography control points based on the distance between UAV 200 and thesubject. For example, computer 220 may determine an aperture size and/orfocal length required to capture an image of the subject at the distancebetween UAV 200 and the subject. Computer 220 may then modify theaperture size and/or focal length of the photography control pointaccording to the determined aperture size and/or focal length.

UAV 10 according to an example embodiment of the present invention isshown in FIGS. 3A-3E. An alternative embodiment, UAV 11, is shown inFIGS. 4A-4D. Many features and components of UAV 10 are similar tofeatures and components of UAV 11, with the same reference numeralsbeing used to indicate features and components that are similar betweenthe embodiments. UAV 10, 11 is used to capture images and may beemployed in a variety of indoor or outdoor applications. In someembodiments, UAV 10, 11 is used to capture images of a subject. Forexample, UAV 10, 11 may be used to capture images of a patient's body orpart thereof. The captured images may be used to detect, monitor,diagnose, and monitor treatment of skin, hair, and/or nail features,conditions, and/or diseases. UAV 10, 11 may be used indoors, for examplein healthcare, professional offices, hospitals, private homes, etc.

UAV 10, 11 is a multirotor drone that is lifted and propelled by rotors(i.e. horizontally-oriented propellers). In the embodiments illustratedin FIGS. 3A-4D, UAV 10, 11 comprises a quadcopter having a body 12, fourarms 14 extending outwardly from the body, and a rotor 16 coupled toeach arm. UAV 10, 11 may comprise any number of rotors capable oflifting and propelling UAV 10, 11. In some embodiments, UAV 10, 11 maycomprise one or more rotor guards. In the embodiment illustrated inFIGS. 3A-3E, UAV 10 comprises rotor guard 18 sized and positioned aboverotors 16 to protect the rotors and/or to prevent rotors 16 from causingdamage should the UAV collide with a user, bystander, and/or otherobject.

In some embodiments, UAV 10, 11 may comprise three, four, or morecoaxial rotors capable of lifting and propelling UAV 10, 11.

To capture images, UAV 10, 11 comprises imaging system 20. Imagingsystem 20 includes at least one camera 30 and at least one light source40. In some embodiments, camera 30 houses at least one light source 40.In the embodiments illustrated in FIGS. 3A-4D, camera 30 is mounted to afront surface 12 a of body 12 and light source 40 is mounted to a bottomsurface 12 c of body 12. Camera 30 and/or light source 40 may be mountedon alternative positions of body 12. In some embodiments, camera 30and/or light source 40 is mounted on body 12 using a gimbal (not shown).The gimbal may permit camera 30 and/or light source 40 to pivot aboutone, two, or three axes. In some embodiments, the gimbal may permitcamera 30 to pivot about one, two, or three axes to reduce the effect ofpropeller vibration on image quality and/or to orient camera 30 forcapturing an image.

Camera 30 is a digital camera that captures high-quality images.High-quality images may be images with a resolution of at least12-megapixels. In some embodiments, images are captured and stored inthe digital memory (e.g. SD card) of camera 30 and/or are captured andwirelessly transmitted to external memory of cloud computing or otherexternal computing devices via WiFi, satellite, and/or mobileconnection. One or more photography parameters of camera 30 such asshutter speed, aperture length, focal length and ISO sensitivity, may beselected such that a desired magnification of an object with minimaloptical distortion is acquired. For example, in some embodiments, thefocal length of camera 30 is between about 15 and about 35 mm and/or theoptical magnification of camera 30 is between about 1.5× and 2×. In someembodiments, camera 30 comprises a 12-megapixel CMOS sensor and 24 mmf/2.8 lens with a 35 mm-equivalent focal length.

Imaging system 20 may comprise one or more optical elements, such aslenses, films, filters, diffusers, and polarizers (e.g. linearpolarizers, circular polarizers, etc.) for improving image qualityand/or for acquiring magnified images. Imaging system 20 may comprise aplurality of lenses. Each lens (not shown) may for example comprise adouble-convex lens, a plano-convex lens, a Fresnel lens, a doublet lens,an achromatic lens, or a meniscus lens. Each lens may be coated with ananti-reflection coating to improve image quality.

In some embodiments, imaging system 20 comprises one or more filters(not shown). The one or more filters may be used to filter and/orpolarize the light emitted by imaging system 20 and/or the light that isreflected by an object or patient to be imaged. In some embodiments, theone or more filters may be used to achieve cross polarization forimproving image quality.

In some embodiments, imaging system 20 comprises multiple cameras havingdifferent optical specifications. For example, FIG. 3F depicts anembodiment of imaging system 20 comprising a first camera 40 a to takeoverview images of a first image quality, a second camera 40 b that hasan optical zoom to take images of a second image quality, and a thirdcamera 40 c to take dermoscopic images. The quality of an image may bethe resolution of the image. The photography parameters may includewhich camera is used to capture a certain image.

Light source 40 may comprise one or more optical elements, such asfilms, filters, diffusers, and polarizers (e.g. linear polarizers,circular polarizers, etc.) for improving image quality and/or foracquiring magnified images. In some embodiments, light source 40includes one or more filters (not shown). The one or more filters may beused to filter and/or polarize the light emitted by light source 40. Insome embodiments, the one or more filters may be used to achieve crosspolarization.

In some embodiments, light emitted by imaging system 20 and/or lightsource 40 illuminates skin to be imaged. Light is reflected by the skinas specular reflection and/or by diffuse reflection. Light rays that arereflected from the surface of an object via specular reflection maycreate glare in the acquired image. Specular reflected light oftencauses the imaged skin to appear shiny. Specular reflected lightinterferes with the acquisition of an image showing detailed features ofthe skin. Specular reflected light tends to have substantially the samepolarization as the incident light emitted by UAV 10, 11. In contrast,diffused light is not polarized. Since skin is partially translucent,some light hitting the surface of the skin is reflected as diffuse lightby the skin's deeper layers. Diffuse light may contain usefulinformation about the skin and its features.

In some embodiments, diffused reflected light passes through one or morefilters (not shown). The one or more filters may be used tosubstantially block specular reflected light rays and/or remove glareand/or acquire a digital image of a feature below the surface of skin.For example, if a first filter is a linear polarizer, then to blockspecular reflected light rays, a second filter may be set with itspolarization axis rotated 90° relative to that of the first filter. Ifthe first filter is a circular polarizer that polarizes light in theclockwise direction, then to block specular reflected light rays, thesecond filter can be a circular polarizer that polarizes light in thecounter clockwise direction. If the first filter is a circular polarizerthat polarizes light in the counter clockwise direction, then to blockspecular reflected light rays, the second filter can be a polarizer thatpolarizes light in the clockwise direction.

In some embodiments, imaging system 20 and/or light source 40 mayinclude a filter (not shown) for providing structured precision lightingto an object to be imaged. Structured lighting may assist in determininga depth of an imaging subject. Structured precision lighting may includeemitting light in a known pattern, for example a regular grid of light.When such emitted light is reflected by an object and captured in animage, the imaged grid may be compared to the emitted grid to determinea depth of the imaging subject.

In some embodiments, UAV 10, 11 may be used to administer photodynamictherapy. For example, light source 40 may emit ultraviolet (UVA and/orUVB) light. Such embodiments may be used to reduce the symptoms ofpsoriasis (e.g. skin pigmentation caused by sun damage).

UVB light may penetrate the skin of a subject and slow the growth ofaffected skin cells. Such phototherapy with UVB light may involveexposing a subject's skin to a UVB light source for a set length of timefor a set period of time.

In some embodiments, a patient may be treated by photodynamic therapyaccording to a photodynamic treatment scheme. System 20 may beconfigured to generate a photodynamic treatment scheme according toimaging of the subject.

To direct UAV 10, 11 about the subject to be imaged, UAV 10, 11 furthercomprises a guidance system. The guidance system may comprise an indoorglobal positioning (indoor-GPS) system. The indoor-GPS system mayprovide accurate location data (e.g. within about 2 cm) for UAV 10, 11.

The indoor-GPS system comprises a transmitter mounted to UAV 10, 11 anda network of stationary receivers (i.e. stations) positioned on theground and/or walls about an object to be imaged by UAV 10, 11. Thetransmitter mounted to UAV 10, 11 and network of stationary receiversare in communication via a radio interface. The indoor-GPS system isused to determine the location of UAV 10, 11 while moving or stationaryby using multiple ranges (i.e. distances) between UAV 10, 11 and thereceivers which are positioned at known locations.

FIGS. 3A-4D illustrate an example embodiment of a localization module ofguidance system 130 comprising an indoor-GPS, wherein the indoor-GPScomprises markers of known locations and camera 70. Camera 70 is mountedto bottom surface 12 c of body 12 and oriented toward the ground. Anetwork of markers is placed on the ground in view of camera 70. Camera70 is positioned and oriented to acquire images of the network ofmarkers positioned on the ground about an object. Camera 70 is a digitalcamera that is used to capture real-time images and/or video of themarkers. Real-time images and/or video may be captured, stored,processed, and analyzed in the digital memory (e.g. SD card) of UAV 10,11 and/or are wirelessly transmitted to an external computing system viaWiFi, satellite, and/or mobile connection. The computing system thenstores, processes, and analyzes the images and/or video to determine aposition of UAV 10, 11.

FIGS. 5A-5B illustrate a network of markers 80 of a localization moduleaccording to an example embodiment. Network of markers 80 comprisesmultiple markers 82 arranged in a grid pattern. In the exampleembodiment shown in FIG. 5A, network 80 comprises a 8×6 grid of markers82 spaced an equidistance apart from one another. However, network 80may comprise any arrangement and configuration of markers 82 providedthat the location of each marker is known.

Each marker 80 in the network of markers 82 has a distinct pattern. Insome embodiments, each marker 80 may comprise a distinct black and whitepattern.

Camera 70 may provide real-time images and/or a video stream of network80 to an external computer system to estimate the position of UAV 10,11. Provided camera 70 is able to image at least one marker 82, theposition of UAV 10, 11 may be estimated. In some embodiments, UAV 10, 11can detect and recognize at least four markers 82.

In some embodiments, UAV 10, 11 is configured to adjust a height(altitude) at which UAV 10, 11 is flying to maintain a certain number ofmarkers 82 in view of camera 70. In some embodiments, UAV 10, 11 must beat least about 30 cm above network 80 to detect at least four markers82.

In flight, UAV 10, 11 uses camera 70 to detect and identify markers 82of network 80. In some embodiments, to detect and identify markers 82,camera 70 automatically extracts contour and/or filters acquired images.In other embodiments, contour extraction and filtering is performed byan external computer and image data is transmitted from camera 70 viaWiFi, satellite, and/or mobile connection.

In some embodiments, each marker 82 comprises a unique ArUco code orQRCode as shown in FIG. 5B. An ArUco code or QRCode consists of blacksquares arranged in a square grid on a white background, a photograph ofwhich can be captured by an imaging device (e.g. camera 70). Thearrangement of black squares in the ArUco code or QRCode may then beextracted from the imaged marker to match the imaged marker with amarker in a database of known markers and marker locations. The matchedmarker may then be used to determine the position of UAV 10, 11 relativeto the ArUco codes.

In some embodiments, a localization module may capture images of network80, and detect contours from the images of network 80. Contours in theimages indicate the presence of squares, and therefore markers 82, inthe images. Images lacking contours may be rejected from furtherprocessing. The black and white squares in each image may then beidentified by dividing the image into cells using horizontal andvertical grid lines. Depending on the amount of black or white pixelspresent in each grid region, each grid region is then assigned a valueof 0 (white) or 1 (black) (or vice versa). The resulting pattern ofblack and white regions in each image is then compared to a database ofblack and white regions for known markers 82 in the network 80 ofmarkers, and the image may be matched to a known marker. The position ofUAV 10, 11 may then be determined from the position of the known marker.The position of UAV 10, 11 may be represented as a translation vectorfrom the known marker, where the translation vector represents adirection and distance of UAV 10, 11 from the known marker.

In some embodiments, four markers 82 may be imaged, and the position ofcamera 70 may be taken as an average of the positions computed accordingto each detected marker 82.

In some embodiments, an adaptive threshold may be used to permit markerdetection in poor light conditions. For example, if less than athreshold number of pixels in an image are designated as white pixels,then a threshold illumination level for identifying a pixel as white maybe lowered. Similarly, if more than a threshold number of pixels in animage are designated as black pixels, then a threshold illuminationlevel for identifying a pixel as black may be increased.

FIGS. 6A-6B illustrate an indoor-GPS 90 according to an exampleembodiment. Indoor-GPS 90 comprises a transmitter 92 mounted to UAV 10,11, at least one receiver 94, and a modem 96. In the embodimentsillustrated in FIGS. 3A-4D, transmitter 92 is mounted to a top surface12 d of body 12 to decrease propeller noise and/or increase the accuracyof navigation. Transmitter 92 may be mounted to alternative positions ofbody 12 provided UAV 10, 11 (and the parts thereof) do not obscure orweaken the strength of the signal emitted by transmitter 92 to a levelwhich cannot be received by receivers 94.

In some embodiments, receivers 94 have an unobstructed line of sight totransmitter 92.

Transmitter 92 emits a signal periodically to provide geolocation andtime information to receivers 94. In some embodiments, transmitter 92emits a signal every 0.1 seconds to 5 seconds. In some embodiments,transmitter 92 emits a signal every 2 seconds.

A time between when transmitter 92 transmits a signal and when each ofreceivers 94 receive the signal is proportional to the distance fromtransmitter 92 to each of receivers 94. Such time delay betweentransmission of the signal and reception of the signal by each ofreceivers 94 may be used to determine the position of transmitter 92,and thereby the position of UAV 10, 11.

In some embodiments, receivers 94 determine a position of transmitter 92by computing one or more navigation equations (e.g. a trilaterationalgorithm) to determine the position of transmitter 92. In someembodiments, indoor-GPS 90 comprises four receivers 94 configured tomeasure four time delays for a signal transmitted by transmitter 92. Insome embodiments, the four time delays are used to compute a system offour equations, wherein the four equations represent three positioncoordinates and a clock deviation. The clock deviation may be used tocorrect for clock deviations between receivers 94.

In some embodiments, indoor-GPS 90 may be configured to autonomouslydetermine a position of each of receivers 94 and determine a map ofreceivers 94. Indoor-GPS 90 may determine a location of receivers 94 byeach of receivers 94 emitting a signal which is then received by eachother of receivers 94. Indoor-GPS 90 may then determine a position ofeach of receivers 94 similar to determining the position of transmitter92 described above. A map of receivers 94 may then be determined fromthe position of each of receivers 94. The map of receivers 94 may bestored in the memory of a modem 96.

Although the embodiment shown in FIG. 6A comprises four receivers 94,more or fewer receivers may be used. One or more receivers 94 may bemounted to one or more walls and/or the ceiling inside a confined space.For example, for the embodiment shown in FIG. 6A, a receiver 94 ismounted to each of the four walls of a rectangular room. Theconfiguration of the walls may take on other geometric shapes (e.g.triangular, square, etc.).

In some embodiments, guidance system 130 uses a library such as the OpenComputer Vision (OpenCV™) for human body skeleton detection andtracking. The input to guidance system 130 may be a video stream, forexample a video stream from imaging system 110. The input video streamis processed frame by frame. A plurality of major body joints of asubject being imaged are identified and tracked by guidance system 130.A skeletal model is then generated from the identified major bodyjoints. The skeletal model may then be used by guidance system 130 tocontrol platform 120, for example to align platform 120 and imagingsystem 110 with an area of interest of a subject. The area of interestmay then be imaged using imaging system 110. By generating the skeletalmodel, guidance system 130 may compensate for unintentional subjectmovement during an imaging session, and/or between imaging sessions.

In some embodiments, UAV 200, 10, 11 comprises one or more laser sensorsto prevent collision of UAV 200, 10, 11 with other objects. The one ormore laser sensors detect objects proximate the UAV 200, 10, 11. If anobject is detected within an threshold distance of UAV 200, 10, 11, UAV200, 10, 11 may autonomously navigate to avoid a collision with thedetected object. In some embodiments, UAV 200, 10, 11 may stop or holdposition when an object is detected within a threshold distance of UAV200, 10, 11.

In some embodiments, UAV 200, 10, 11 comprises one or more laser sensorsfor determining the height of the UAV relative to the ground.

Analysis system 140 may comprise computer software to acquire, store,process, manage, and/or manipulate digital images. In some embodiments,the software may be stored on UAV 200, 10, 11, a mobile device carriedby UAV 200, 10, 11, and/or a computer in communication with UAV 200, 10,11, for example computer 220. The software may be used to improve imagequality. For example, the software may be used to control illuminationand/or colour, bring an object to be imaged into focus, and/or correctimage defects (for example, by making corrections for artifacts such asoil or gel bubbles, hair, and/or shadows). The software may use agraphics processing unit and/or central processing unit to processimages in real-time.

Where UAV 200, 10, 11 is used to digitally image skin, the software maybe used to label, archive, monitor, and/or analyze skin featuresincluding, but not limited to, lesions, psoriasis, eczema, wounds, andwrinkles. For example, the software may be used to monitor changes inthe height, diameter, and/or pigmentation of such skin features bycomparing two or more digital images acquired at different times. Insome embodiments, the appearance and/or disappearance of skin featuresmay be monitored over subsequent images.

In some embodiments, the software is configured to process image data tocalculate an ABCD (i.e. “Asymmetry, Border, Colors, and Dermoscopicstructures”) score and/or other conventional dermoscopic criteria. Suchprocessing may be used to analyze skin lesions such as pigmented andnon-pigmented lesions. For example, the automated analysis of the datacaptured may be used to determine if a lesion is prone to be benign ormalignant growth and if further treatment and examination isrecommended. The software may also recommend a personalized skin careand/or treatment plan. The software can also generate a report to besent to a specialist for further examination and monitoring.

The software may provide a database of images for comparison anddiagnostic purposes. Diagnosis may be performed automatically by thesoftware and/or performed by a user or the user's physician.

In some embodiments, UAV 200, 10, 11 may be configured to capturemultiple images of an object from different viewpoints to construct athree-dimensional (3D) reconstruction of the object. FIG. 6C depicts anexample embodiment of 3D reconstruction method 602.

The input of 3D reconstruction algorithm 602 is a set of images 604captured from different angles with varying degree of overlap. In firststep 606 of the 3D reconstruction algorithm, a set of sparse key pointsis generated for each image in the set of overlapping images, and a setof feature descriptors (compact numerical representations) is generatedfrom the set of sparse key points.

In some embodiments, the sparse key points may comprise corner pointsdetermined using one or more methods described in U.S. Pat. No.6,711,293 titled Method and apparatus for identifying scale invariantfeatures in an image and use of same for locating an object in an image,or in Fast explicit diffusion for accelerated features in nonlinearscale spaces by Pablo F Alcantarilla, Jesús Nuevo, and Adrien Bartoli.(Trans. Pattern Anal. Machine Intell, 34(7):1281-1298, 2011), herebyincorporated by reference.

In second step 608 of the 3D reconstruction algorithm, photometricmatches between each image are determined based on a number of featuredescriptors in common between two images. An Euclidean distance fromeach feature descriptor to each other feature descriptor is determined,and the Euclidean distances are compared to determine an initial set ofphotometric matches comprising the best matching images.

In step 610, two images are selected from the initial set of bestmatching images as an initial baseline from which to construct aninitial sparse 3D point-cloud. The two initial images may be selectedbased on a number of corresponding feature descriptors.

In step 612, the initial sparse 3D point-cloud is then iterativelyextended by adding images from the set of images by using poseestimation and triangulation, for example using an incremental structurefrom motion (SfM) algorithm. Key point matches may be removed which havesimilar descriptors but are incorrect in their geometric location withrespect to other key point matches.

Once the sparse 3D point-cloud is generated, the position of the camerain space may be estimated in step 614, for example by using a methoddescribed in Adaptive structure from motion with a contrario modelestimation by Pierre Moulon, Pascal Monasse, and Renaud Marlet. (ACCV,2012), hereby incorporated by reference herein.

From the sparse 3D point-cloud and position of the camera, a rough dense3D mesh may be generated in step 616. A smooth refined 3D mesh may thenbe generated from the rough dense 3D mesh in step 618. Finally, thesmooth refined 3D mesh is textured in step 620 using the initial set ofimaged to generate a textured 3D model.

In some embodiments, the dense point-cloud may be generated according toa method described in PatchMatch: A randomized correspondence algorithmfor structural image editing by Barnes, C., Shechtman, E., Finkelstein,A. and Goldman, D. B. (ACM Transactions on Graphics (ToG), 28(3), 2009),hereby incorporated by reference.

In some embodiments, the smooth refined 3D mesh may be generatedaccording to a method described in High accuracy andvisibility-consistent dense multiview stereo by Vu, H. H., Labatut, P.,Pons, J. P. and Keriven, R. (IEEE transactions on pattern analysis andmachine intelligence, 34(5)), hereby incorporated by reference.

In some embodiments, the UAV 200, 10, 11 may acquire an image of a skinlesion of a subject and map the lesion image to a previously identifiedskin lesion on a body map of the subject. UAV 200, 10, 11 may useautomated or supervised pattern-matching to map the skin lesion in theimage to a previously identified skin lesion. The image may be a lowerquality overview image acquired using a digital camera, or higherquality dermoscopy image acquired using a dermoscope.

FIG. 7A is a schematic top view of an embodiment of system 100comprising an unmanned ground vehicle (UGV) 700, and FIG. 7B is aschematic front view of UGV 700. UGV 700 comprises body 710. UGV 700 ispropelled by two or more wheels 720. Wheels 720 are powered by one ormore motors 730 mounted to body 710.

To navigate UGV 700 about a subject, UGV 700 further comprises aguidance system. The guidance system comprises an indoor globalpositioning system (indoor-GPS). The indoor-GPS may provide accuratelocation data for UGV 700, similar to the indoor-GPS system of UAV 10,11 as described above.

UGV 700 further comprises digital camera 740, light source 750, sensors760, and controller 770. Controller 770 is communicatively coupled todigital camera 740, light source 750, sensors 760, and motors 730.

FIG. 7 illustrates an embodiment of UGV 700 wherein sensor 760 comprisesa camera 761 of the localization module. Camera 761 is positioned andoriented to acquire images of a network of markers positioned on theground about an object.

Light source 750 is configured to illuminate a region proximate to UGV700, and digital camera 740 is configured to capture images of subjectsilluminated by light source 750.

In some embodiments, instead of digital camera 740, UGV 700 may comprisea mount configured to receive a computing device comprising an imagingsystem, for example a tablet computer comprising a camera such as anApple iPad™′.

FIG. 8 is a schematic view of an embodiment of system 100 comprisingcircular stand 800. Circular stand 800 comprises body 810 mounted onpedestal 820. Pedestal 820 is mounted to track 830. Track 830 follows asubstantially circular path about center 840. Pedestal 820 comprisesmotor 822 for propelling pedestal 820 along track 830.

Digital camera 850, light source 860, and sensors 870 are mounted onbody 810. Light source 860 is configured to illuminate a subject locatedat center 840, and digital camera 850 is configured to photograph asubject at center 840 illuminated by light source 860.

Circular stand 800 is communicatively coupled to control system 880.Control system 880 may be communicatively coupled by a wired and/orwireless connection to digital camera 850, light source 860, sensors870, and motor 822. Control system 880 receives sensor data from sensors870 and digital images from camera 850. Control system 880 may receiveuser input, and/or access stored digital images. Stored digital imagesmay include digital images previously taken of a subject.

Control system 880 controls circular stand 800 based on one or more of:sensor data received form sensors 870, digital images received fromdigital camera 850, user input, and stored digital images.

FIG. 9A depicts a schematic view of an embodiment of imaging systemcomprising flash light case 900. A flash light case is hand-held devicecomprising one or more light sources and a mount for an imaging device.The light sources of the flash light case are configured to illuminate aregion proximate the flash light case, and the mount of the flash lightcase is configured to orientate an imaging device towards the regionilluminated by the light sources when an imaging device is mounted onthe mount.

Flash light case 900 comprises body 902, light source 904 and mount 906.Light source 904 and mount 906 are attached to body 902. Mount 906 isconfigured to receive a computing device comprising an imaging system,for example a tablet computer comprising a camera such as an AppleiPad™′.

Flash light case 910 further comprises handles 912 attached to body 902.Handles 912 support body 902. Handles 912 are configured to be graspedby a user of flash light case 910.

FIG. 9B depicts computing device 908 received by mount 904. Computingdevice 908 comprises digital camera 910. As depicted in FIG. 9B, whencomputing device 908 is received by mount 904, digital camera 910 isoriented towards a subject illuminated by light source 904. By graspinghandles 912, a user may move flash light case 900 about a subject, andorientate light source 904 and a digital camera 910 relative to thesubject.

Computing device 908 may be configured to provide commands to a user offlash light case 910 for moving flash light case 910 and controllingdigital camera 910. For example, computing device 908 may generate acommand to move flash light case 910 to orientate digital camera 910relative to a feature of a subject, and photograph the feature withdigital camera 910.

Interpretation of Terms

Unless the context clearly requires otherwise, throughout thedescription and the claims:

-   -   “comprise”, “comprising”, and the like are to be construed in an        inclusive sense, as opposed to an exclusive or exhaustive sense;        that is to say, in the sense of “including, but not limited to”;    -   “connected”, “coupled”, or any variant thereof, means any        connection or coupling, either direct or indirect, between two        or more elements; the coupling or connection between the        elements can be physical, logical, or a combination thereof;        elements which are integrally formed may be considered to be        connected or coupled;    -   “herein”, “above”, “below”, and words of similar import, when        used to describe this specification, shall refer to this        specification as a whole, and not to any particular portions of        this specification;    -   “or”, in reference to a list of two or more items, covers all of        the following interpretations of the word: any of the items in        the list, all of the items in the list, and any combination of        the items in the list;    -   the singular forms “a”, “an”, and “the” also include the meaning        of any appropriate plural forms.

Words that indicate directions such as “vertical”, “transverse”,“horizontal”, “upward”, “downward”, “forward”, “backward”, “inward”,“outward”, “vertical”, “transverse”, “left”, “right”, “front”, “back”,“top”, “bottom”, “below”, “above”, “under”, and the like, used in thisdescription and any accompanying claims (where present), depend on thespecific orientation of the apparatus described and illustrated. Thesubject matter described herein may assume various alternativeorientations. Accordingly, these directional terms are not strictlydefined and should not be interpreted narrowly.

Software and other modules may reside on servers, workstations, personalcomputers, tablet computers, image data encoders, image data decoders,PDAs, color-grading tools, video projectors, audio-visual receivers,displays (such as televisions), digital cinema projectors, mediaplayers, and other devices suitable for the purposes described herein.Those skilled in the relevant art will appreciate that aspects of thesystem can be practised with other communications, data processing, orcomputer system configurations, including: Internet appliances,hand-held devices (including personal digital assistants (PDAs)),wearable computers, all manner of cellular or mobile phones,multi-processor systems, microprocessor-based or programmable consumerelectronics (e.g., video projectors, audio-visual receivers, displays,such as televisions, and the like), set-top boxes, color-grading tools,network PCs, mini-computers, mainframe computers, and the like.

The invention may also be provided in the form of a program product. Theprogram product may comprise any non-transitory medium which carries aset of computer-readable instructions which, when executed by a dataprocessor, cause the data processor to execute a method of theinvention. Program products according to the invention may be in any ofa wide variety of forms. The program product may comprise, for example,non-transitory media such as magnetic data storage media includingfloppy diskettes, hard disk drives, optical data storage media includingCD ROMs, DVDs, electronic data storage media including ROMs, flash RAM,EPROMs, hardwired or preprogrammed chips (e.g., EEPROM semiconductorchips), nanotechnology memory, or the like. The computer-readablesignals on the program product may optionally be compressed orencrypted.

In some embodiments, the invention may be implemented in software. Forgreater clarity, “software” includes any instructions executed on aprocessor, and may include (but is not limited to) firmware, residentsoftware, microcode, and the like. Both processing hardware and softwaremay be centralized or distributed (or a combination thereof), in wholeor in part, as known to those skilled in the art. For example, softwareand other modules may be accessible via local memory, via a network, viaa browser or other application in a distributed computing context, orvia other means suitable for the purposes described above.

Specific examples of systems, methods and apparatus have been describedherein for purposes of illustration. These are only examples. Thetechnology provided herein can be applied to systems other than theexample systems described above. Many alterations, modifications,additions, omissions, and permutations are possible within the practiceof this invention. This invention includes variations on describedembodiments that would be apparent to the skilled addressee, includingvariations obtained by: replacing features, elements and/or acts withequivalent features, elements and/or acts; mixing and matching offeatures, elements and/or acts from different embodiments; combiningfeatures, elements and/or acts from embodiments as described herein withfeatures, elements and/or acts of other technology; and/or omittingcombining features, elements and/or acts from described embodiments.

It is therefore intended that the following appended claims and claimshereafter introduced are interpreted to include all such modifications,permutations, additions, omissions, and sub-combinations as mayreasonably be inferred. The scope of the claims should not be limited bythe preferred embodiments set forth in the examples, but should be giventhe broadest interpretation consistent with the description as a whole.

While a number of exemplary aspects and embodiments are discussedherein, those of skill in the art will recognize certain modifications,permutations, additions and sub-combinations thereof.

Various features are described herein as being present in “someembodiments”. Such features are not mandatory and may not be present inall embodiments. Embodiments of the invention may include zero, any oneor any combination of two or more of such features. This is limited onlyto the extent that certain ones of such features are incompatible withother ones of such features in the sense that it would be impossible fora person of ordinary skill in the art to construct a practicalembodiment that combines such incompatible features. Consequently, thedescription that “some embodiments” possess feature A and “someembodiments” possess feature B should be interpreted as an expressindication that the inventors also contemplate embodiments which combinefeatures A and B (unless the description states otherwise or features Aand B are fundamentally incompatible).

While a number of exemplary aspects and embodiments have been discussedabove, those of skill in the art will recognize certain modifications,permutations, additions and sub-combinations thereof. It is thereforeintended that the following appended claims and claims hereafterintroduced are interpreted to include all such modifications,permutations, additions and sub-combinations as are within their truespirit and scope.

1. A method of photographing at least a portion of a subject with a platform carrying an imaging system, the method comprising: generating a photography scheme, the photography scheme comprising a set of photography control points, each of the photography control points comprising: a location of the platform relative to the subject; an orientation of the platform relative to the subject; and one or more photography parameters; determining a location and an orientation of the platform carrying the imaging system; navigating the platform to each of the photography control points and operating the imaging system to capture an image of the subject at each of the photography control points according to the associated photography parameters.
 2. The method according to claim 1, wherein generating the photography scheme based at least in part on the location and orientation of the subject comprises: determining a distance between the subject and the platform for each of the photography control points; and determining an orientation of the platform relative to the subject for each of the photography control points.
 3. The method according to claim 1, comprising: receiving a previously captured image of the subject; determining one or more photography parameters associated with the previously captured image; and wherein generating the photography scheme comprises generating one or more photography control points with photography parameters equal to the photography parameters associated with the previously captured image.
 4. The method according to claim 3, comprising: determining a location and an orientation of the platform relative to the subject associated with the previously captured image; and wherein generating the photography scheme comprises generating a photography control point with: a location of the platform relative to the subject equal to the location of the platform relative to the subject associated with the previously captured image; and an orientation of the platform relative to the subject equal to the orientation of the platform relative to the subject associated with the previously captured image. 5-6. (canceled)
 7. The method according to claim 1, comprising determining a location and an orientation of the subject, and wherein capturing an image of the subject at each of the photography control points comprises determining a position and an orientation of the platform at each of the photography control points at least in part based on the location and orientation of the subject.
 8. The method according to claim 7, wherein determining the location and the orientation of the subject comprises retrieving a location and an orientation of a representative subject.
 9. The method according to claim 7, wherein determining the location and the orientation of the subject comprises: capturing one or more images of the subject with the imaging system; and determining the location and the orientation of the subject from the images.
 10. The method according to claim 1, wherein navigating the platform to each of the photography control points comprises: determining a current position of the platform; determining a translation vector representing a direction and distance from the current position of the platform to one of the photography control points; and navigating the platform for the distance and in the direction of the translation vector.
 11. The method according to claim 1, wherein determining the location of the platform comprises: capturing an image of a marker; matching the image of the marker to a known marker in a set of known markers, wherein each of the known markers is associated with a corresponding marker location; and determining the position of the platform from the known marker and corresponding marker location.
 12. The method according to claim 11, wherein each of the known markers is associated with a corresponding marker orientation, and determining the orientation of the platform comprises: determining an orientation of the marker in the image of the marker; and determining the orientation of the platform from the orientation of the marker in the image of the marker and the corresponding marker orientation of the known marker.
 13. The method according to claim 1, wherein determining the location and the orientation of the platform comprises: transmitting a signal from the platform; receiving the signal with three receivers with known locations; determining time delays of the signal to the three receivers; and determining a position of the platform from the time delays and known locations of the receivers.
 14. The method according to claim 1, wherein: a smartphone comprising a digital camera is mounted to the platform; and the imaging system comprises the digital camera of the smartphone. 15-16. (canceled)
 17. The method according to claim 1, wherein: the platform comprises a propulsion system; and the platform is at least partially self-propelled by the propulsion system.
 18. The method according to claim 17, wherein: a smartphone comprising a digital camera is mounted to the platform; the imaging system comprises the digital camera of the smartphone; the smartphone is communicatively coupled to the platform to control the propulsion system; and navigating the platform to each of the photography control points comprises the smartphone controlling the propulsion system to navigate the platform to each of the photography control points. 19-21. (canceled)
 22. The method according to claim 18, wherein the smartphone comprises a memory and the photography scheme is stored in the memory of the smartphone.
 23. The method according to claim 22, wherein navigating the platform to each of the photography control points comprises executing guidance software with the smartphone stored in the memory of the smartphone. 24-25. (canceled)
 26. The method according to claim 1, wherein: the platform comprises an aerial drone; the imaging system comprises a digital camera mounted to the aerial drone; and navigating the platform to each of the photography control points comprises flying the aerial drone to each of the photography control points.
 27. The method according to claim 1, wherein: the platform comprises an unmanned ground vehicle (UGV); the imaging system comprises a digital camera mounted to the UGV; and navigating the platform to each of the photography control points comprises driving the UGV between each of the photography points. 28-45. (canceled)
 46. An imaging system comprising: an unmanned aerial drone, the drone comprising: a drone body; four rotors mounted to the drone body; a digital camera mounted to the drone body; a light source mounted to the drone body; a laser sensor mounted to the drone body; a drone transceiver mounted to the drone body; a drone computer mounted to the drone body, the drone computer configured to: control the four rotors to navigate the drone; control the digital camera to capture one or more digital images; control the light source to emit light; receive data from the laser sensor; and transmit and receive data via the drone transceiver; three GPS receivers, wherein each of the GPS receivers is configured to receive a signal from the drone transceiver; a controller, the controller comprising: a memory storing at least one previous image of a subject; a controller transceiver configured to communicate with the drone transceiver and the three GPS receivers; wherein the controller is configured to control the drone to: control the rotors to navigate the drone about a subject; control the rotors to orientate the digital camera towards the subject; control the light source to illuminate the subject; control the digital camera to take one or more images of the subject; and store the one or more images of the subject in the memory; wherein the controller is configured to: generate a photography scheme, the photography scheme comprising a set of photography control points, each of the photography control points comprising: a location of the drone relative to the subject; an orientation of the drone relative to the subject; and one or more photography parameters; determine a location and an orientation of the drone from the signal received by the GPS receivers; navigate the drone to each of the photography control points and operate the digital camera and light source to capture an image of the subject at each of the photography control points according to the associated photography parameters.
 47. (canceled)
 48. The system according to claim 46, wherein the controller is configured to: determine a location and an orientation of the drone associated with the previous image of the subject; and wherein generating the photography scheme comprises generating a photography control point with: a location of the drone relative to the subject equal to the location of the drone relative to the subject associated with the previous image of the subject; and an orientation of the drone relative to the subject equal to the orientation of the drone relative to the subject associated with the previous image of the subject. 49-81. (canceled) 